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This  research  program  has  been  directed  at  discovering  neurobioiogical  features  (cell 
specializations,  activity  patterns  in  collections  of  neurons,  circuit  design  features)  essential 
to  the  encoding,  organizing,  and  utilizing  recognition  memories  by  networks  in  mammalian 
cortex.  Three  levels  of  work  have  been  involved:  1)  physiological  and  anatomical  analyses 
of  specific,  well  defined  examples  of  telencephalic  networks;  2)  translation  of  the 
neurobioiogical  results  into  computer  simulations  of  the  networks;  3)  testing  of  the 
physiological  and  behavioral  predictions  of  the  computer  models.  It  is  anticipated  that 
the  results  of  the  research  will  provide  the  bases  of  formal  hypotheses  regarding  the  links 
between  cortical  properties  and  computational  operations  and  provide  guidelines  for  the 
design  of  novel  devices  capable  of  dealing  with  the  difficult  computational  problems 
presented  by  real  world  environments.  .The  following  points  summarize  the  progress  made 
towards  these  goals  during  the  tenure  of  ONR  Grant  #N00014-86-K-0333.  .  _ 

(1)  Links  were  discovered  between  learning  related  brain  rhythms  and  the  cellular 
machinery  that  causes  stable  changes  in  synaptic  strength.  These  relationships  were 
converted  into  a  set  of  biologically  valid,  synaptic  "learning  rules"  for  implementation  in 
neural  networks.  These  rules  describe  where  and  when  changes  will  occur  on  cells 
receiving  complex  spatio-temporal  patterns  of  input  activity.  The  observation  that 
rhythmic  activity  has  a  profound  impact  on  the  operations  of  cortical  networks  directly 
influenced  nearly  all  aspects  of  the  modelling  efforts  that  followed. 

(2)  The  above  discoveries  were  made  in  hippocampus,  a  structure  in  many  ways  ideal 
for  neurobioiogical  work  but  less  than  appropriate  as  a  starting  point  for  building 
computer  simulations  with  predictive  power  (sec  original  proposal  for  arguments  relating 
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to  this  point).  The  olfactory  bulb-olfactory  (piriform)  cortex  was  selected  as  a  target  for 
modelling  and  it  was  essential  to  establish  that  the  rules  developed  for  hippocampus 
extend  to  this  system.  Experimental  work  showed  that  the  basic  cell  biology  and 
spatio-temporal  patterns  involved  in  modifying  hippocampus  held  true  for  cortex  as  well. 
Differences  are  present  and  the  origins  and  significance  of  these  arc  still  under  study.  A 
particularly  noteworthy  point  established  by  the  neurobiological  work  is  that  the  basic 
sampling  rhythm  for  olfaction  (4-7  Hz)  is  uniquely  effective  in  changing  synaptic 
strength;  this  provides  a  connection  between  behavior  and  specific  cellular  phenomena. 

(3)  Two  further  tests  were  made  of  the  behavioral  significance  of  the  phenomena  and 
rules  obtained  from  the  neurobiological  studies.  First,  the  same  rhythmic  patterns  of 
stimulation  used  to  probe  the  networks  in  vitro  were  employed  as  discriminative  cues  in 
an  olfactory  learning  problem.  Rats  learned  and  remembered  these  "electric  odors”  about 
as  well  as  they  did  natural  odors.  Moreover,  the  same  synaptic  changes  found  in  the  in 
vitro  experiments  developed  in  the  freely  moving  animals  as  learning  occurred.  Studies 
are  still  in  progress  on  these  phenomena.  Second,  drugs  that  block  the  neurobiological 
mechanisms  that  produce  synaptic  modifications  also  block  learning,  and  do  so  without 
impairing  recall.  These  results  give  us  confidence  that  the  biological  properties  we  have 
identified  are  in  fact  critical  to  the  formation  of  recognition  memories. 

(4)  Computer  simulations  of  the  bulb-cortex  system  have  been  found  to  possess 
unexpected  capacities  for  analyzing  input  stimuli  and  for  organizing  memories.  Following 
learning  of  many  cues,  the  model  is  able  to  recognize  a  complex  (known)  signal  even 
when  it  is  masked  by  a  much  stronger  input.  It  does  this  by  exploiting  repetitive 
sampling  and  various  inhibitory  processes  (simulations  of  inhibitory  potentials  found  in 
cortical  networks)  it  contains.  Perhaps  more  remarkably,  the  simulation  also  builds 
perceptual  hierarchies  without  outside  supervision.  As  noted,  the  model  uses  repetitive 
sampling.  In  our  earlier  studies,  we  found  that  after  extensive  learning  the  model  would 
generate  a  single  response  pattern  (i.c.,  collection  of  cortical  cells  becoming  active)  to 
closely  related  signals  in  the  stimulus  world;  responses  on  subsequent  sample  cycles  were 


unique  to  the  cue  now  present.  Recently,  we  have  found  that  the  combined  bulb-cortex 
network  will  perform  true  hierarchical  clustering  in  that  it  will  form  representations  for 
groups,  subgroups,  and  individuals.  This  is  a  task  that  is  computationally  difficult  and 
one  of  considerable  practical  significance.  We  have  extracted  an  alogrithm  from  a 

simplified  version  of  the  network  and  found  it  to  be  novel,  parallel,  and  to  show 

temporal  linearity  with  regard  to  the  number  of  network  elements  and  input  signals.  Thus 
the  model  may  be  appropriate  for  silicon  implementation  and  will  scale  upwards  with 

linear  cost.  We  are  currently  exploring  engineering  possibilities. 

(5)  The  first  tests  of  the  predictions  of  the  model  have  been  carried  out  using 

chronic  recording  from  the  olfactory  cortex  of  rats  engaged  in  learning  an  odor 

discrimination  problem.  These  experiments  have  confirmed  that  the  cortex  uses  sparse 

coding  (i.e.,  few  cells  respond  to  any  given  odor)  and  that  its  activity  is  synchronized 
with  the  sampling  rate.  This  work  is  still  in  progress  and  more  detailed  tests  should  be 
available  shortly. 

1.  Physiological  studies  of  svnaptic  learning  rules. 

Hippocampal  long  term  potentiation  serves  as  a  model  synaptic  plasticity  mechanism 
for  rapid  acquisition  and  long-term  retention  of  memories  for  "facts"  or  "data".  Our 
studies  have  aimed  at  identifying  naturally-occurring  cell  discharge  pattens  that  induce 
LTP  and  uncovering  the  cellular  mechanisms  responsible  for  the  efficacy  of  these 
patterns.  The  hippocampal  EEG  during  learning  exhibits  a  sinusoidal  4-7  Hz  EEG  wave 
(theta)  with  short  bursts  of  cell  discharges  rising  on  the  peaks  of  the  waves.  Our  prior 
work  established  that  brief  bursts  of  afferent  stimulation,  synchronized  to  the  frequency 
of  the  theta  rhythm  (5  Hz)  are  optimal  for  induction  of  LTP  in  field  CA1.  We  have  now 

shown  that  this  "theta  burst”  pattern  is  particularly  effective  because,  by  reducing 

synaptic  inhibition,  it  promotes  levels  of  postsynaptic  depolarization  sufficient  to  allow 
calcium  influx  through  NMDA  receptor-linked  channels  (Larson  and  Lynch,  1988).  The 
voltage  dependence  of  LTP  induction  proves  to  be  a  significant  parameter  in  the 
operation  of  the  piriform  model  (sec  below).  Our  prior  work  indicated  that  a  burst  of 
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activity  in  one  input  to  a  postsynaptic  target  cell  docs  not  induce  LTP  but  "primes"  the 
cell  so  that  a  burst  to  a  separate  input  one  theta  cycle  later  does  induce  LTP.  This 
provides  one  temporal  rule  for  synaptic  interactions  and  LTP.  We  have  extended  these 
studies  to  examine  interactions  between  inputs  within  one  theta  cycle.  A  "primed"  cell 
received  asynchronous  bursts  to  three  separate  inputs  such  that  stimulation  of  the  second 
input  temporally  overlapped  stimulation  of  the  first  and  third  but  the  first  and  third  did 
not  overlap  with  each  other.  Most  LTP  was  induced  at  synapses  activated  by  the  first 
burst,  less  by  the  second,  and  least  by  the  third.  This  effect  involves  both  retrograde 
facilitation  and  anterograde  suppression  effects  within  one  theta  cycle  (Larson  and  Lynch, 
1989;  Greenberg  et  al.,  1988).  Indeed,  it  appears  that  induction  of  LTP  at  one  set  of 
synapses  is  followed  by  a  brief  refractory  period  when  LTP  at  other  synapses  is  inhibited 
(Greenberg  et  al .,  1988)  In  summary,  this  set  of  studies  accomplished  two  objectives. 
One,  it  identified  for  the  first  time  a  link  between  brain  wave  patterns  and  mechanisms 
that  produce  synaptic  changes.  Two,  it  provided  a  set  of  biologically  valid  synaptic 
learning  rules  for  computer  modelling. 

Chronic  recording  techniques  have  been  used  to  address  two  questions  related  to  the 
stability  of  LTP.  In  the  first,  we  simply  asked  how  long  the  potentiation  effect  lasts. 
Rats  were  chronically  implanted  with  stimulation  electrodes  in  the  Schaffer-commissural 
system  and  recording  electrodes  in  the  CA1  field.  Responses  were  tested  for  several  days 
before  and  1-4  weeks  after  induction  of  LTP  by  theta  burst  stimulation  (TBS).  Once 
induced,  LTP  was  found  to  persist  in  non-decremental  fashion  for  as  long  as  recording 
conditions  could  be  maintained;  in  some  cases  this  was  several  weeks  (Staubli  and  Lynch, 
1987).  Given  that  LTP  appears  to  persist  indefinitely,  the  question  arises  as  to  whether 
or  not  particular  patterns  of  synaptic  activity  might  actively  reverse  it.  LTP  was  induced 
by  theta  burst  stimulation  in  the  chronic  preparation  and  shortly  thereafter  (5-10  min)  the 
potentiated  input  was  stimulated  at  low  frequency  (l-5Hz  for  50-100  sec).  In  the  majority 
of  animals  (64%)  low  frequency  stimulation  reversed  the  LTP  when  the  animals  were 
tested  cither  1  hr  or  1  day  later.  Subsequent  burst  stimulation  established  that  LTP  could 
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be  reinstated  in  these  animals.  Low  frequency  stimulation  did  not  reduce  responses  on 
inputs  that  had  not  been  potentiated  (Staubli  and  Lynch,  in  press).  These  findings 

suggest  that  LTP  can  be  persistent  enough  to  encode  long-term  memories  but  may  be 
subject  to  reversal  shortly  after  its  induction. 

Ongoing  studies  are  aimed  at  understanding  the  cellular  basis  of  the  theta-plasticity 
rules  and  the  mechanism  by  which  LTP  can  be  reversed.  In  terms  of  building 

simulations,  precise  data  concerning  spatial  and  temporal  interactions  between  synapses  are 
crucial  and  these  learning  rules  arc  being  incorporated  into  current  model. 

2.  Piriform  physiology,  in  vitro. 

The  synaptic  learning  rules  developed  thus  far  are  based  on  the  CA1  field  of 

hippocampus.  For  a  variety  of  reasons  including  more  direct  access  to  peripheral  input, 
the  model  we  are  using  for  simulation  work  is  of  the  piriform  cortex.  Therefore,  we 
have  been  studying  the  basic  physiology  of  this  region  with  particular  regard  to  plasticity 
mechanisms.  Field  potentials  evoked  in  the  superficial  layers  by  stimulation  of  the  lateral 
olfactory  tract  (LOT)  or  associational  (ASSN)  inputs  to  piriform  cortex  in  vitro  can  be 
distinguished  by  three  criteria:  (1)  visual  placement  of  the  stimulation  electrode  in  either 
the  LOT  or  in  layer  lb,  (2)  laminar  profile  analysis,  and  (3)  absence  of  paired-pulse 
facilitation  at  ASSN  synapses. 

Our  first  experiments  were  to  establish  the  receptor  pharmacology  of  these  two 
systems  and  compare  them  to  hippocampus.  In  the  presence  of  physiological  levels  of 

extracellular  magnesium,  we  found  that  the  NMDA  antagonist  APS  (50  uM),  Kad  no 
detectable  effect  on  synaptic  responses  in  either  system  but  the  non-NMDA  antagonist 
DNQX  (20  uM)  completely  suppressed  responses  in  both  systems.  However,  in 

magnesium-free  medium  both  systems  exhibited  NMDA  receptor-mediated  responses  as 
demonstrated  either  by  the  presence  of  a  response  in  DNQX  r .■  the  blockade  of  a 
response  component  in  AP5  (Larson  el  al.,  1989). 

For  LTP  experiments,  two  preparations  were  used.  One  of  "rostral"  piriform, 
containing  the  LOT  and  one  of  "caudal"  piriform,  beginning  at  the  point  where  the  LOT 
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ceases  to  be  visible  as  a  definable  fiber  tract.  Theta  burst  stimulation  to  the  LOT 
induced  LTP  in  16/48  slices  examined,  a  proportion  much  lower  than  that  observed  in 
hippocampal  experiments.  Stimulation  of  ASSN  inputs  in  rostral  slices  induced  LTP  in 
16/30  cases  but  in  36/40  caudal  slices.  In  both  systems,  TBS  did  not  induce  the 
immediate,  decrcmcntal  short-term  potentiation  observed  in  hippocampus,  rather 
potentiation  developed  gradually  over  1-2  minutes  after  TBS.  The  low  success  rate  at 
LOT  synapses  raised  the  possibility  that  the  NMDA  system  might  be  difficult  to  activate 
in  these  contacts.  Stimulation  with  long  100  Hz  trains  was  equally  ineffective  in  slices 
that  failed  to  potentiate  after  TBS;  however,  TBS  applied  in  Mg++-frce  medium,  a 
condition  which  greatly  facilitates  NMDA  receptor  currents,  induced  LTP  in  5/5  slices 
tested.  The  pharmacology  of  LTP  induction  was  also  studied  in  the  caudal  associational 
system  where  LTP  induction  was  very  reliable.  Perfusion  with  AP5  blocked  LTP 
induction  in  9/9  slices  and  LTP  was  obtained  after  drug  washout  in  6/6  cases  (Jung, 
Larson,  and  Lynch,  in  prep.). 

These  results  indicate  that  both  LOT  and  ASSN  synapses  in  piriform  cortex  possess 
the  NMDA  receptor-linked  LTP  induction  mechanism  and  that  it  can  be  activated  by  TBS. 
This  encourages  the  speculation  that  the  LTP  rules  derived  from  hippocampus  hold  for 
piriform  as  well,  but  demonstration  of  this  requires  further  experiments. 

3.  Piriform  synaptic  plasticity  in  vivo. 

Rats  sample  odors  at  the  theta  rhythm  during  mystacial  sniffing  and  it  is  reasonable 
to  assume  that  signals  are  processed  by  the  olfactory  system  at  this  rhythm.  Indeed,  even 
the  hippocampal  BEG  is  synchronized  to  the  sniff  cycle.  We  have  developed  a  paradigm 
to  test  learning-induced  changes  in  synaptic  efficacy  in  the  piriform  cortex  using  theta 
burst  stimulation  of  the  LOT  as  an  "electric  odor"  discriminative  stimulus.  Rats  were  first 
trained  on  a  two  odor  successive  cue  go-no  go  task  for  water  reward.  After  learning-set 
acquisition,  LOT  stimulation  was  substituted  for  the  positive  odor.  Single-pulses  at  the 
theta  frequency  did  not  elicit  any  obvious  behavioral  reaction,  but  when  theta  bursts  were 
used  most  rats  began  sniffing  and  approached  the  water  port  as  they  do  in  response  to 
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the  positive  odor.  They  rapidly  learned  to  discriminate  between  the  electric  odor  and 

real  odors  as  well  as  between  two  separate  electric  odors.  Examination  o'  single-pulse 
responses  evoked  by  the  electrode  used  as  a  cue  revealed  a  substantial  synaptic 

potentiation  (35%)  after  a  single  training  session.  This  effect  was  still  evident  24  hrs. 

later.  Interestingly,  the  same  stimulation  pattern  did  not  induce  potentiation  when  given 
to  naive  animals,  suggesting  that  some  other  system  that  facilitates  synaptic  plasticity  is 
activated  during  learning  sessions  (Roman  et  al.,  1987).  This  may  also  relate  to  the 
difficulty  in  obtaining  LTP  in  the  LOT  synapses  in  slices.  In  any  case,  bursts  of  activity 
occurring  at  the  frequency  of  behavioral  sniffing  which  is  synchronized  to  the 

hippocampal  EEG  proves  to  be  an  effective  sensory  cue  and,  moreover,  results  in  synaptic 
potentiation  in  piriform  as  in  hippocampus.  This  provides  a  further  link  between  the 
LTP-based  learning  rules  derived  from  hippocampus  and  endogenous  rhythms  and 
behavior. 

4.  Simulation  and  theoretical  analysis  of  piriform  cortex 

a)  Hierarchical  readout  via  repetitive  sampling 

Simulation  and  analysis  of  layer  II  of  piriform  cortex  have  incorporated  a  broad 
range  of  its  anatomical  and  physiological  characteristics  in  an  attempt  to  understand  how 
these  properties  might  interact  to  produce  coherent  operation,  and  how  these  operations 
might  generate  recognizable,  useful  and  testable  computational  and  psychological  function. 
Two  networks,  bulb  and  cortex,  consisting  of  distinct  architectures  and  physiologies,  are 
extensively  connected  via  both  feedforward  and  feedback  projections.  The  entire  system 
works  in  synchrony  with  a  4-7  Hz  (theta)  sampling  pattern  that  is  characteristic  of  small 
mammals.  Bulb  mitral  cells  (these  neurons  innervated  by  the  peripheral  receptors  and 
that  project  to  cortex)  receive  inputs  presented  repetitively  for  brief  periods.  Inputs  to 
the  cortical  network  arise  from  the  resultant  synchronous  bursting  in  a  subset  of  mitral 
cells,  yielding  cyclic  activity  in  relatively  discrete  "operation  cycles"  time-locked  to  the 
sampling  rhythm.  Sparse  random  connectivity  in  the  simulation  selectively  activates  those 
cortical  cells  whose  dendrites  arc  most  connected  to  the  input  lines  that  arc  active. 
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Learning  increments  active  synapses  on  sufficiently  depolarized  cells  via  an  LTP  rule. 
Input  lines  shared  across  many  similar  input  cues,  and  thus  participating  in  many 
learning  episodes,  will  strengthen  their  target  synapses  more  than  lines  which  participate 
in  relatively  fewer  episodes.  The  result  is  that  cortical  dendrites  (which  can  be  viewed 
as  vectors  being  moved  by  synaptic  learning)  become  increasingly  well-tuned  to  those 
inputs  containing  the  shared  subset;  i.c.,  those  inputs  that  are  sufficiently  similar  to 
constitute  members  of  a  cluster.  We  have  shown  that  this  circuit  will  generate  cell-firing 
responses  that  group  learned  cues  by  similarity.  That  is,  for  a  given  threshold  of  input 
similarity  among  a  set  of  cues,  outputs  are  identical  for  all  of  the  cues,  whereas  below 
that  similarity  threshold,  outputs  are  much  less  similar  than  corresponding  inputs 
(Granger,  Ambros-Ingerson  and  Lynch,  1989).  The  predominantly  feedforward  collateral 
axons  of  the  cortex  enhance  this  effect  (Granger,  Whitson  and  Lynch,  1989). 

Feedback  from  cortex  to  the  bulb  inhibitory  layer  in  the  model  is  trained  via  a 
correlational  rule  during  an  earlier  "developmental"  period.  The  feedback  then  selectively 
inhibits  those  bulb  mitral  cells  that  are  most  responsible  for  the  cortical  output  response, 
via  relatively  long-lasting  inhibition.  Resulting  renormalization  of  bulb  activity  produces 
a  distinct  spatial  pattern  of  bulb  mitral  cell  firing,  which  in  turn  activates  a  distinct  set 
of  cortical  cells.  This  inhibitory  feedback  process  can  continue  until  bulb  is  sufficiently 
inhibited  to  be  largely  quiescent.  Cortical  responses  after  the  initial  (first  sample) 
response  become  progressively  more  different  for  different  cues,  increasingly  approaching 
unique  encodings  of  individual  cues. 

Taken  together,  the  resulting  sequence  of  responses  are  hierarchically  ordered  such 
that  the  first-cycle  response  indicates  similarity-based  cluster  information  for  learned  cues 
whereas  subsequent  cycle  responses  denote  increasingly  subordinate  categories  (Granger, 
Ambros-Ingerson,  Henry  and  Lynch,  1989;  Granger,  Ambros-Ingerson  and  Lynch,  1989). 
These  findings  suggest  that  sensory  cues  may  be  "iteratively  recognized"  at  a  sequence  of 
successively  lower  levels,  with  the  first  level  recognized  corresponding  to  a  natural  "entry 
level"  for  perceptual  processing.  Cognitive  studies  of  visual  and  conceptual  recognition  in 
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hierarchically  organized  domains  indicate  that  such  preferred  recognition  levels  do  exist 
and  exert  a  strong  influence  over  early  processing:  human  subjects  robustly  prefer  a  “basic 
level"  description  (c.g.,  "bird"  in  the  hierarchy  animal-bird-sparrow),  recognizing  cues  faster 
at  this  level  and  more  frequently  than  either  superordinate  or  subordinate  referents  for 
the  same  objects.  We  have  found  empirically  that  first-cycle  responses  from  the  piriform 
network  robustly  indicate  the  basic  level  as  empirically  identified  in  human  experiments 
using  the  same  hierarchically-organized  data  for  both  the  human  subjects  and  the 
simulation  (Granger,  Gluck,  Crane  and  Lynch,  1989).  The  network  first-cycle  responses  at 
the  basic  level  are  followed  by  subordinate  level  information  in  second  and  later  cycles, 
separated  by  intervals  of  200  msec  simulated  time  in  the  model;  interestingly,  studies  by 
Hoffmann  and  Ziessler  (1983)  indicate  that  average  reaction  time  differences  between 
recognition  of  the  basic  and  immediately-subordinate  description  levels  was  148  msec, 
b)  Information-theoretic  view  of  hierarchical  clustering 

We  have  also  found  that  a  proposed  mathematical  predictor  of  the  basic  level  effect 
successfully  predicts  both  the  human  and  network  performance  on  this  data  (Gluck  and 
Corter,  1985;  Lynch,  Granger,  Larson  and  Baudry,  1989).  This  predictor  is  an 
information-theoretic  measure  of  the  information  value  of  clustering  of  data  into  groups, 
measuring  the  increased  information  about  features  of  a  cue  gained  by  knowledge  of  its 
category  membership.  Briefly,  for  cues  considered  as  multidimensional  vectors,  if  a 
particular  cue  dimension  D  can  take  on  values  Vj  ...rn  with  probabilities  p{v j)  ...  p(rn), 
respectively,  then  the  uncertainty  of  the  value  of  D  is: 

U(D)  =  ~£p(v,)logp(v,) 

«= i 

(Shannon  1948),  and  this  uncertainty  about  the  value  of  D  for  this  cue  can  be  reduced, 
via  knowledge  that  the  cue  is  in  a  particular  category  C,  by  the  amount  U(D)  -  U(D/C) 
where 

n 

U(D\C)  =  -  52 p(vi\C)  log p(vi\c) 

i  =  l 

and  p(v-JC)  is  the  conditional  probability  of  attribute  value  Vj  given  that  the  cue  is  in 
category  C.  Thus  a  measure  of  the  overall  information  value  /  of  a  category  is  the 
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expected  reduction  in  the  uncertainty  of  the  value  of  O  due  to  knowledge  of  category 
membership: 


/ 


P(C){U(D)  -  U(D\C)) 


P(C) 


71 

-  5ZKt,i)l°g  P(V>)  - 
1  =  1 


p(v,\C) 


The  information-theoretic  origin  of  the  measure  suggests  that  the  clustering  performed  by 
the  piriform  network  may  be  learning  information-optimal  clusters;  this  idea  is  being 
tested  on  a  range  of  hierarchically-organized  data  by  measuring  the  information  value  of 
the  simulation-generated  clusters  and  comparing  them  against  optimal  values  (see  Lynch, 
Granger,  Larson,  and  Baudry  (1989)  for  discussion),  and  analyzing  sensitivity  of  these 
results  to  perturbations  in  parameters  of  the  model. 

A  given  cue  population  can  be  divided  into  clusters  in  factorially  many  ways  (see 
Granger  et  al.,  1988;  Lynch,  Granger,  Larson  &  Baudry,  1989);  Granger  Ambros-Ingerson 
and  Lynch  (1989)  empirically  and  analytically  related  synaptic  parameters  of  the  cortical 
simulation  to  its  clustering  performance,  deriving  a  monotonic  relation  between  the  ratio 
of  potentiated  (maximum)  to  naive  (minimum)  synaptic  weights,  and  the  breadth  (maximal 
distance  between  members)  of  the  clusters  formed  from  a  given  cue  population.  A 
developmental  algorithm  was  proposed  to  sample  the  cue  environment  and  set  these  weight 
parameters  so  as  to  automatically  give  rise  to  information-optimal  clustering  of  the  cues 
in  the  "adult"  network  (Granger,  Ambros-Ingerson  and  Lynch,  1989). 

c)  LTP  induction  rules,  layer  III,  and  sequence  learning 

Simulation  experiments  have  thus  far  focused  primarily  on  the  expression 
characteristics  of  LTP,  embedded  in  a  specific  layer  of  cortex  (layer  II),  emphasizing  the 
sequential  memory  encoding  and  readout  described.  Initial  experiments  with  the  induction 
mechanisms  of  LTP  indicate  that  the  "sequence"  rule  described  earlier,  in  combination 
with  rules  for  the  interaction  of  layer  III  cells  and  the  basal  dendrites  of  layer  II  and  III 
cells,  enables  encodings  that  powerfully  distinguish  among  cues  consisting  of  slightly 
different  sequences  of  cue  constituents  (c.g.  ABC,BCA),  which  might  occur  given  different 
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relative  concentrations  of  chemicals,  or  different  orders  of  frequencies  or  formant 
transitions  occurring  within  a  phoneme.  In  addition,  modeling  efforts  aimed  at  the 
functional  implications  of  a  temporary,  non-synapsc-spccif ic  plasticity  mechanism  like  that 
of  potentiation  of  the  dentate  mossy  fiber  synapses  on  hippocampal  CA3  pyramidal  cells, 
and  interactions  between  this  short-term  potentiation  and  long-term  mechanisms  (e.g.,  via 
via  known  feedback  pathways  from  hippocampus  to  olfactory  bulb),  suggest  that  these 
mechanisms  might  combine  to  enable  learning  of  longer  temporal  sequences  of  cues. 

d)  A  novel  algorithm  for  hierarchical  clustering 

The  ability  of  the  network  to  identify  hierarchical  statistical  structure  in  data  raises 
questions  of  characterization  of  the  essential  design  features  of  the  network  underlying  its 
hierarchical  clustering  ability.  Analysis  has  recently  led  to  such  a  characterization,  and 
controlled  testing  revealed  that  the  resulting  simplified  formation  of  the  network  provides 
a  novel  and  efficient  algorithm  for  hierarchical  clustering. 

The  simplified  formulation  consists  of  a  weight  matrix  W  (corresponding  to  the  layer 
I  connectivity  matrix;  see  Fig.  1)  divided  into  H  non-overlapping  "winners-take-all"  or 
"competitive"  patches:  sets  Pv  P2,  ...,  PH  of  weight  vectors  C  (i.e.,  columns  of  HO  such 
that  W  =  Uj/’j.  The  network  is  trained  on  a  set  of  IV-dimensional  real-valued  vectors  (i.e., 
spatial  patterns  of  activation  in  the  lateral  olfactory  tract),  via  an  extension  of  a 
correlational  (Hebbian)  learning  algorithm  (a  simplification  of  the  LTP  rules  for  synaptic 
modification).  For  each  input  vector  X ,  the  column  vectors  (dendrites)  C  that  win  the 
(winners-take-all)  competition  on  X  (corresponding  to  target  calls  that  are  most 
depolarized  by  this  input)  are  identified.  The  synaptic  contacts  on  these  vectors  are  then 

trained,  moving  the  vectors  closer  to  X  by  an  increment  7c-  Feedback  from  the 
just-trained  vectors  then  partially  inhibits  or  ‘masks’  the  input.  The  remainder  of  the 
input  is  presented  to  the  next  network  patch  matrix  in  the  hierarchy,  until  all  H 
hierarchical  subnets  have  been  trained,  over  H  operation  cycles.  At  any  given 
hierarchical  level,  the  Cs  can  be  shown  to  converge  to  the  means  of  the  clusters  of  cues 
on  which  they  arc  trained,  as  in  related  "competitive  learning"  algorithms  (e.g.,  von  dcr 


Malsburg,  1973;  Grossbcrg,  1976;  Kohoncn,  1984;  Rumclhart  and  Zipscr,  1986).  The 
feedback  inhibition  step  enables  vectors  in  IV  assigned  to  the  subordinate  hierarchical 
levels  to  converge  to  means  of  subclustcrs  of  the  data,  allowing  secondary  (and  //-ary) 
structure  to  be  identified  (for  //  divisions  of  the  weight  matrix). 

Formally: 

for  A’  in  training-sample 
for  h  £  {1,2  ....  //} 
for  C  6  win  (A”,  I\) 

C  ^C  +  lc(X  -  C) 
end-for 

X  <—  X  -  mean(win(A',  P^)) 
end-for 

end-for  (1) 

where  H  is  the  depth  of  the  hierarchy;  C  is  the  subset  of  weight  vectors  in  a  patch  Ph  that  wins 
the  competition  on  the  input  X;  win(AT,  Ph)  =  {C  €  Ph  :  X  -  C  =  ma xc,ePh(X  •  C,)  A  X  C  >  0}; 
and  7 c  is  the  learning  rate. 

Testing  has  shown  that  this  novel  network  algorithm  robustly  identifies  hierarchical 
statistical  structure  in  data,  ana  that  its  results  compare  well  with  those  for  standard 
hierarchical  clustering  techniques.  Moreover,  the  algorithm  is  inherently  efficient,  and 
scales  linearly  with  the  size  of  the  cue  population  to  be  clustered.  In  particular,  the 
space  complexity  of  the  algorithm  is  O(n),  i.e.,  the  number  of  nodes  required  for 
completion  separation  of  n  cues  depends  only  linearly  on  n;  and  training  time  per 
presentation  is  also  linear  in  the  total  number  of  categories  of  cues  to  be  distinguished 
(Ambros-Ingcrson,  Granger  and  Lynch,  1989). 

Simulation  and  theoretical  analysis  of  this  cortical  network  has  thus  provided  an 
instance  in  which  a  novel  and  efficient  algorithm  for  a  well-studied  computational 
problem  has  been  developed  from  a  specific  cortical  network.  Reflecting  the  system  from 
which  it  was  derived,  the  algorithm  is  inherently  parallel  and  hence  lends  itself  to 
implementation  in  silicon.  A  collaborative  effort  with  Dan  Ha  mmerstrom  of  Adaptive 
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Systems,  Inc.,  is  undertaking  the  design  of  a  novel  chip  ("silicon  cortex")  based  on  the 
network  analysis  described  here.  Bailey  and  Hammerstrom  (1988)  have  shown  the  need 
for  multiplexing  of  interconnections  in  silicon  implementations  of  networks  of  sizes  on 
the  order  of  hundreds  of  thousands  of  cells,  via  analysis  of  the  space  constraints  involved 
in  available  CMOS  processes,  and  have  designed  a  novel  technique  (Augmented  Broadcast 
Hierarchy)  for  this  purpose,  which  is  being  used  in  the  implementation  of  this  cortical 
network. 

5.  Sinele-unit  recording  in  piriform  cortex  In  behaving  rats. 

Recordings  of  single-cell  activity  during  olfactory  behavior  are  used  to  test  the 
validity  of  the  model.  Animals  are  trained  in  a  two-odor  successive  cue  go-no  go 
olfactory  discrimination  task.  After  about  5  sessions  using  different  odor  pairs,  the  rats 
acquire  a  new  discrimination  within  about  five  trials.  Single-units  in  piriform  are 
recorded  using  tungsten  microelectrodes  attached  to  a  moveable  microdrive  implanted  to 
the  skull.  Once  a  cell  is  isolated,  the  animal  is  run  on  several  training  sessions  using 
novel  or  previously-learned  odor  pairs. 

To  date  we  have  recorded  activity  in  some  60  well-isolated  units.  On  the  basis  of 
mean  firing  rate,  these  units  fall  into  two  obviously  different  categories.  Type  I  units 
are  the  most  commonly  encountered  (55/60),  have  rather  slow  discharge  rates  (1-5  Hz), 
broad  spikes  (>500  msec  duration  of  negativity,  filtered),  and  occasionally  emit  a  high 
frequency  burst  of  2-12  spikes  (100  Hz).  Type  II  units  are  rare  (5/60),  fire  rapidly  (20-50 
Hz),  and  have  narrow  spikes  (<500  msec  negativity). 

Depending  on  how  long  the  unit  can  be  held,  firing  patterns  are  recorded  during  2  to 

20  sessions  with  different  odor  pairs.  Over  half  (24/43)  type  I  units  did  not  respond  to 

any  of  the  odors  tested,  typically  four  sessions  (8  odors).  Eight  units  showed  selective 

firing  to  only  one  of  the  odors  tested.  The  rest  responded  with  increased  firing  to  more 

than  one  odor.  The  characteristic  response  of  a  sensitive  unit  consisted  of  a  single  spike 
or  a  short  burst  on  the  majority  of  trials  when  that  odor  was  presented.  The  response 
typically  occurred  250  msec  after  the  valve  was  switched  to  present  the  odorant. 
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Some  of  the  type  II  units  we  have  thus  far  recorded  have  shown  interesting 
properties.  Activity  of  these  rapidly  firing  cells  shows  a  marked  suppression  (sometimes 
near  complete  silence  after  firing  at  20-40  Hz)  when  the  rat  enters  the  sniff  port 
(odor  delivery  area)  but  before  odor  onset.  Upon  odor  presentation,  the  cells  resume 
firing,  sometimes  beginning  with  a  "rebound  burst”.  Since  we  suspect  these  neurons  to  be 
inhibitory  interncurons,  this  may  indicate  a  suppression  of  inhibition  while  the  animal  is 
awaiting  the  olfactory  cue.  However,  it  is  clear  that  we  need  to  record  many  more  of 
these  cells  to  verify  these  conclusions. 

Although  these  results  are  clearly  preliminary,  and  more  data  is  needed,  it  is  clear 
that  piriform  units  are  not  responding  in  global  fashion  to  any  odor  presented.  Over  75% 
of  the  type  I  cells  tested  responded  to  less  than  two  of  the  array  of  odors  presented. 
The  anatomy  and  connectivity  of  piriform  as  well  as  the  operation  of  the  network  model 
suggest  sparse  coding  of  odors  and  our  preliminary  results  suggest  this  to  be  the  case. 

6.  Studies  of  olfactory  behavior:  LTP  and  memory. 

Behavioral  studies  are  used  to  test  predictions  of  the  model  and  examine  how 
higher-order  effects  can  arise  from  the  simpler  functions  achieved  by  the  network. 
Results  from  the  simulation  suggest  that  the  capacity  of  the  olfactory  memory  system 
should  be  large  and  hence  that  representations  for  learned  odors  should  not  suffer 
interference  from  new  learning  and  that  mixtures  of  novel  odors  should  be  treated 
differently  than  mixtures  of  previously  learned  odors.  Also  of  interest  is  how  other  brain 
networks,  such  as  the  dorsomedial  nucleus  and  hippocampus  interact  with  piriform  in 
olfactory  memory  storage. 

One  key  assumption  we  have  made  is  that  LTP  is  used  as  the  learning  mechanism  for 
representational  forms  of  memory.  We  have  tested  this  by  asking  whether  or  not  LTP 
and  memory  share  a  common  pharmacology.  That  is,  if  LTP  is  involved  in  memory 
encoding,  it  is  expected  that  drugs  which  disrupt  LTP  should  also  prevent  acquisition  of 
new  representations.  The  NMDA  receptor  is  a  logical  place  to  start  since  it  is  crucial  to 
LTP  induction  and  potent  and  selective  antagonists  for  the  receptor  arc  available.  Wc 
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had  previously  shown  that  intraventricular  infusion  of  AP5  blocks  LTP  in  vivo  and  also 
prevents  acquisition  of  spatial  representations  (maps)  required  for  navigation  in  a  water 
maze  (Morris  et  at .,  Nature  319:  774-776,  1986);  we  have  now  tested  the  effects  of  this 
manipulation  on  olfactory  discrimination  learning.  We  used  a  two-odor 
simultaneous-presentation  olfactory  discrimination  task.  Rats  were  trained  on  8  pairs  of 
odors  prior  to  AP5  treatment.  After  AP5,  acquisition  of  new  discriminations  was 
impaired  although  memory  for  previously  learned  odors  was  intact  (Staubli  et  al.,  1989). 

We  had  also  previously  shown  that  intra-ventricular  administration  of  leupeptin,  a 
protease  inhibitor,  blocks  both  spatial  (Staubli  et  al.,  Behav.,  Neural.  Biol.,  40:58-69,  1984) 
and  olfactory  (Staubli  et  al..  Brain  Res.  337:  333-336)  learning.  We  have  now  completed 
two  studies  testing  the  effects  of  leupeptin  on  LTP  induction.  In  the  first,  the  drug  (20 
mg/ml)  was  chronically  infused  into  the  lateral  ventricles  of  rats  prepared  with  chronic 
stimulation  and  recording  electrodes  in  the  CA1  field.  Leupeptin  infusion  had  very  little 
effect  on  baseline  evoked  responses  but  when  LTP  was  tested  (3-5  days  after  beginning 
infusion)  only  3  of  13  animals  showed  potentiation  one  day  later  and  only  one  exhibited 
stable  potentiation  for  several  days.  Control  animals  with  saline  pumps  all  (11  of  11) 
exhibited  robust  and  stable  LTP.  The  block  of  LTP  by  leupeptin  was  reversible:  after 
disconnection  of  the  pumps,  LTP  was  induced  in  6  of  7  cases  (Staubli  et  al.,  1988).  In 
the  second  study,  hippocampal  slices  were  incubated  in  the  presence  of  40-100  uM 
leupeptin.  Incubation  for  2  hr  or  less  with  the  drug  had  no  effect  on  LTP  but  after  3 
hrs  or  more,  LTP  was  significantly  reduced  (Oliver  et  al.,  1989).  Examination  of  the 
postsynaptic  responses  to  the  theta  burst  stimulation  used  to  induce  LTP  indicated  that 
leupeptin  had  no  significant  effects  on  the  depolarization  and  presumably  the  NMDA 
response  in  the  slices  in  which  LTP  was  reduced.  Studies  using  recently  developed  and 
more  selective  calpain  inhibitors  are  presently  in  progress. 

It  should  be  noted  that  neither  AP5  nor  leupeptin  affected  acquisition  of  avoidance 
conditioning.  Current  concepts  emphasize  the  existence  of  multiple  memory  forms  using 
different  brain  systems;  this  may  be  a  reflection  of  these  distinctions. 
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Wc  have  conducted  a  number  of  studies  examining  the  nature  of  the  olfactory 
memory  system.  As  noted  above,  others  have  shown  that  rats  acquire  learning-sets  for 
odor  discriminations  and  this  is  dependent  on  the  dorsomedial  nucleus  of  the  thalamus. 
Once  the  learning-set  is  formed,  rats  learn  new  discriminations  after  as  few  as  one 
exposure  to  each  of  an  odor  pair.  Training  on  as  many  as  30  odor  pairs  was  found  not 
to  impair  subsequent  learning  of  new  pairs,  indicating  that  the  capacity  of  the  system  is 
substantial  and  that  already  established  representations  do  not  interfere  with  learning  of 
new  ones  (Staubli  et  al.,  1987).  These  results  are  in  accord  with  findings  from 
experiments  using  the  piriform  network  simulation.  Tests  using  novel  odor  pairs  composed 
of  smell  mixtures  having  common  component  odorants  indicated  that  rats  learn  these  as 
gestalts  rather  than  analyzing  the  components,  also  in  agreement  with  the  simulation. 
However,  when  a  previously-learned  odor  was  included  in  a  composite  odor  of  reversed 
valence,  rats  were  impaired  as  if  they  were  perceiving  the  known  smell  rather  than  a 
novel  one  (Staubli  et  al.,  1987).  This  result  also  has  an  analog  in  the  model. 

Influence  of  the  hippocampus  on  olfactory  learning  was  tested  by  removing  its 
primary  input,  the  entorhinal  cortex.  Entorhinal  lesions  resulted  in  rapid  forgetting  of 
olfactory  discriminations  -  that  is,  a  discrimination  was  acquired  normally  during  a 
training  session,  but  no  significant  retention  was  displayed  3  hrs  later  (Staubli  et  al., 
1986).  These  results  imply  that  a  system  activated  by  the  hippocampus,  perhaps  involving 
the  medial  septum-diagonal  bands  complex,  is  necessary  for  long-term  retention. 
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